The present invention relates to a method for protecting a video frame sequence against random and/or burst packet loss, in particular to a method of video stream protection in IP network with packet loss.
In modern world exponential IP traffic growth can be observed, because Voice & Video over IP (VVOIP) systems usage expansion. In such condition insufficient network capacity causes network impairments, e.g. packet loss, delays and network jitter which dramatically decrease video and audio quality in real-time VVoIP systems especially for video transmission over an IP network. Most popular modern video codecs such as H.264 AVC, H.264 SVC and HEVC use differential coding algorithms and thus, if even a single video frame is lost during transmission, then all subsequent video frames cannot be decoded until periodic INTRA frame reception. Even low network packet loss causes strong video quality degradation because of this reason. So, video traffic protection algorithms development for IP networks with packet loss is a very important problem. Robust video traffic protection algorithms allow significantly improving the video quality in IP networks with packet loss.
A modern video stream packet loss protection algorithm should satisfy the following requirements: Minimizing time interval between correctly decoded video frames after frame loss, allowing fast decoder state recovery after a packet loss, minimizing coding efficiency degradation, minimizing frame rate degradation, avoiding to introduce additional delay at the sender and receiver sides and minimizing recovery error propagation if a lost frame cannot be exactly restored. The algorithm should protect the video stream from random and burst packet losses because modern routers can use different algorithms for congestion control.
Lost packet retransmission (NACK) as described in RFC 4585: “Extended RTP Profile for Real-time Transport Control Protocol (RTCP)-Based Feedback (RTP/AVPF)” is the most popular method used for packet loss protection. According to this method, the receiver detects a packet loss and sends a corresponding negative acknowledgment to the transmitter side. The transmitter resends a lost packet to the receiver based on this information. This algorithm is efficient only for network channels with low round-trip time and low packet loss probability. Otherwise, it introduces a significant algorithmic delay and thus dramatically decreases the video quality. This method is external with respect to the video codec and the amount of requested information doesn't depend on the video content. It reduces coding efficiency with respect to methods which integrate source and channel coding. Forward error correction is often used for lost packet recovery in network with big network delay and jitter, but the usage of this method for video stream protection is limited by a low FEC efficiency for the case where each video frame is encoded separately because of short frame length in RTP packets or by a significant algorithmic delay for the case where several video frames are encoded by one code word and by a long algorithmic delay if RTP packet interleaving is used.
External methods of video stream packet loss protection which use interleaved source coding add redundancy at the source coding level for frame loss protection. FIG. 14 shows a flow chart for interleaved source coding. According to FIG. 14, interleaved source coding introduces two or more interleaved video streams 1601, 1602 which are encoded independently. If one video stream 1601 cannot be decoded because of packet loss, other video streams 1602 can be decoded and displayed 1603. All video streams can be restored after a GOP period, when a periodic INTRA frame arrives. Interleaved source coding is an effective method for video stream packet loss protection. It can be used for random and burst packet loss protection, but it has the following drawback. If even a single packet in one independent stream is lost then this steam cannot be recovered until periodic INTRA frame receiving. In this case, the displayed frame rate is decreased and the time distance between two correctly decoded frames is increased on the receiver side. After packet loss, the correct receiver state can be restored only after a periodic INTRA frame receiving, so INTRA frames should be inserted sufficiently often and consequently, for a constant bit rate video quality will be deteriorated.
Video redundancy coding as described by S. Wenger, G. Knorr, J. Ott, and F. Kossentini in “Error resilience support in H.263+,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 8, pp. 867-877, November 1998, extends the ISC algorithm. As in the ISC algorithm, the source video stream is separated into two interleaved video streams 1701, 1702 according to FIG. 15 which streams are independently coded. For a fast decoder state recovery, special synchronization frames 1703, 1705 are introduced. These frames are predicted from both video streams 1701, 1702 and can be decoded if at least one independent video stream is uncorrupted. After the synchronization frame independent video streams are predicted from this frame. The VRC algorithm allows decreasing the decoder state recovery time relative to the general ISC algorithm. The VRC algorithm however has a lot of drawbacks. If an INTRA frame is used as synchronization frame, then this algorithm has the same drawbacks as the general ISC algorithm. If an SP frame is used as synchronization frame, then it should require less bits than an INTRA frame, but if several video frame streams are used, the synchronization frame will require much more bits and thereby synchronization frame encoding efficiency will be decreased. Further, the VRC algorithm is sensitive to synchronization frame loss. If this frame is lost, then all subsequent video frame streams cannot be decoded. Only forward prediction is used for synchronization frame encoding. It reduces coding efficiency relative to a bidirectional frame prediction. After a packet loss the video decoder state can only be recovered after synchronization frame receiving. It thus decreases the decoder state recovery time relative to other packet loss protection algorithms.